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Abstract 

In 1997 the British Columbia Open University (BCOU) adopted a virtualization 
strategy based primarily on twinning off-line independent study distance 
education courses (textbook-based with study guide and telephone and e-mail 
tutor support) with alternative online versions (textbook-based with integrated 
conferencing and communications provided through a learning management 
system). The intent was two-fold: (a) To increase course completion rates to British 
Columbia (BC) system levels, and (b) To establish a strong online presence, 
rapidly and cost effectively. 'Course-twinning' also provided an opportunity to 
assess the impact of online delivery on independent study course completion 
rates. In this study we benchmarked successful completion rates (SCRBs) for 
BCOU's online academic courses and compared those to off-line course 
benchmarks. 

Online SCRBss are consistently higher than off-line SCRBs. Online SCRBs gains 
follow a clear pattern: they increase with course level and subject matter 
classifications and are particularly large at the upper-level and in more 
quantitative disciplines (up to 21.2% increase). Online successful completion rates 
are comparable to BC system performance statistics in the credit course areas but 
fall well short for Adult Basic Education (ABE) courses. Since evidence from a 
contemporary course survey indicates that BCOU student participation in online 
course versions is not linked to demographic characteristics associated with 
successful completion rates, we suggest that BCOU student success online results 
primarily from the benefits of online course delivery. We discuss the implications 
of these results. 

Resume 

En 1997, TOpen University de Colombie-Britannique (BCOU) adoptait une 
strategie de virtualisation basee principalement sur Taction de jumelage des cours 
hors-ligne a distance (bases sur un livre ave un guide d'etude et du soutien tutoral 
par telephone ou courriel) avec des versions alternatives en ligne (bases sur un 
livre avec de la visioconference et des outils de communication integres fournis 
grace a un systeme de gestion de Tapprentissage). Cette strategie etait a la base de 
deux intentions : a) Augmenter les taux de completion des cours pour atteindre 
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ceux du systeme de la Colombie-Britannique, et b) Etablir une forte presence en 
ligne rapidement et a pen de frais. Le jumelage des cours permettait aussi 
d'evaluer Timpact de la diffusion en ligne sur les taux de completion des cours 
hors-ligne. Dans cette etude, nous avons etalonne les taux de completion reussie 
(SCRBs) pour les cours academiques en ligne de BCOU et les avons compares a 
I'etalonnage des cours hors-ligne. 

Les SCRBs des cours en ligne sont systematiquement plus haut que ceux des cours 
hors-ligne. Les gains des SCRBs des cours en ligne suivent un pattern clair : ils 
augmentent avec les classifications du niveau et du sujet du cours et sont 
particulierement eleves aux niveaux superieurs et dans les disciplines plus 
quantitatives (jusqu'a 21,2 % d' augmentation). Les taux de completion reussie en 
ligne sont comparables a ceux des statistiques de performance du systeme de la 
Colombie-Britannique pour les cours credites, mais sont de beaucoup inferieurs 
pour les cours de base en education des adultes. Etant donne que les resultats 
d'un sondage recent indiquent que la participation des etudiants aux cours en 
ligne n'est pas liee a des caracteristiques demographiques, nous suggerons que le 
succes des etudiants de BCOU en ligne resulte principalement des benefices de la 
diffusion en ligne des cours. Nous discutons les implications de ces resultats. 


Introduction 

The British Coiumbia Open University (BCOU) Course Compietion Chailenge 

Historically, distance education course completion rates have been low 
compared to 'traditional' face-to-face courses (Beaffy-Guenfer 2001, Carr 
2000, Holmberg 1995, Howell ef ah, 2004, Keegan 1986, Munro 1991, Paul 
1986, 1990, Zajkowski 1997). When in 1997 fhe Open Learning Agency 
(OLA), BCOU's parenf insfifufion, published preliminary research 
findings suggesfing fhaf BCOU's online independenf sfudy course 
complefion rafes were 10 percenfage poinfs higher fhan fheir off-line 
counferparfs (Eliesen and Black 1997, 1998a, 1998b), online course 
delivery became cenfral fo BCOU's goal of increasing course complefion 
rafes "by an amounf sufficienf fo ensure fhaf wifhin fwo years (i.e., by 
March 1999) fhey meef or exceed fhe minimum as esfablished by fhe ofher 
insfifufions in fhe British Columbia College, Insfifufe and Agency sysfem 
(confrolling for program area and using sysfem-wide key performance 
indicators.)" OLA (1997, p. 3, as ouflined in OLA's Ends Policies. 1997 fo 
2000 .) 

To achieve fhis Ends Policy, fhe BCOU adopted a virfualizafion 
sfrafegy based on fwo componenfs: (a) Supplemenfing ifs off-line courses 
wifh e-mail communicafions, and (b) Converfing off-line independenf 
sfudy courses (fexfbook-based wifh insfrucfional resources, and 
felephone and e-mail fufor supporf) wifh fwin online course versions 
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(textbook-based with instructional resources and integrated conferencing 
and communications provided through a learning management system), 
intending for sfudenfs fo choose befween an online and an off-line course 
version for as many courses as possible. "Course-fwinning" ensured fhaf 
sfudenfs who did nof have access fo, or did nof wish fo use online 
fechnology, refained access fo an equivalenf off-line (prinf-based) course 
version in ferms of fexfbooks, assignmenfs, exams, marking guides, 
fufors, credif fransfer, efc. (See more defails below) This approach was 
expecfed fo increase course complefion while profecfing enrolmenf since 
fhe flexible course delivery feafures valued by open learning sfudenfs 
were refained: open admission, confinuous enrolmenf, flexible sfarf and 
end dales, generous complefion and exfension periods, efc. (Black ef ak, 
1994; Black, 1995; Black, 1996; Bufler & Bischoff, 1999; Giguere, 2002). To 
achieve fhis oufcome in a cosf-effecfive manner, fhe insfifufion developed 
software solufions such as "Oscar" (Klassen ef ak, 1999; Porfer, 2001) and 
"Trucourse" (Friesen ef ak, 2007) fhaf 'aufomafically'oufpuffed course 
assignmenfs and course unifs fo bofh off-line and online formafs. 
Furfhermore, by re-purposing exisfing off-line insfrucfional resources 
fhrough Oscar and Trucourse, fhe insfifufion endeavoured fo esfablish a 
sfrong online presence rapidly and cosf effectively. 

The BCOU as a Natural Experiment 

Because of ifs course-fwinning virfualizafion sfrafegy and because BCOU 
assefs were frozen befween 2002 and 2005 in preparation for BCOU's 
acquisifion by anofher insfifufion, fhe BCOU represenfs a nafural 
experimenf fo assess fhe impacf of online delivery on independenf sfudy 
course complefion rales. BCOU academic policies and procedures, 
curriculum, sfaff and faculfy, budgef, and business developmenf acfivifies 
were frozen befween February 2002 and April 2005, and fhe universify 
came as close as is possible fo fhe scientific experimenfal moffo of 
'holding everyfhing else consfanf'. Since fhis period of sfabilify occurred 
while fhe BCOU was abouf half way fhrough implementing ifs course- 
fwinning sfrafegy, many of ifs courses exisfed in 'equivalenf' fwin online 
and off-line versions. By benchmarking BCOU online and off-line 
independenf sfudy course complefion rales during fhis period using a 
sfandard approach (aufhor, 2007), fhis article assesses fhe impacf of online 
delivery on independenf sfudy course complefion rales and checks on 
BCOU's progress foward ifs goal of increasing course complefion rales fo 
BC sysfem levels. 

The British Coiumbia Open University 

In Canada, education is a provincial or ferriforial jurisdiction. Af fhe 
beginning of fhis millennium, fhe Province of British Columbia's Minisfry 



52 


THE IMPACT OF ‘VIRTUALIZATION’ 


of Advanced Education funded and supported 25 postsecondary 
institutions of which the BCOU was a constituent member. At that time, 
BCOU's course enrolment ranged between 20-25,000 and it served 12- 
14,000 students annually before it ceased to operate on March 31, 2005; its 
operations were then taken over by Thompson Rivers University (TRU). 

In its final days, the BCOU was a comparable institution to 
TeleUniversite (Quebec) and Athabasca University (Alberta). It was a 
single mode distance education institution offering flexible learning 
opportunities through credit banking and distance education. BCOU's 
course enrolment was slightly less than TeleUniversite and significantly 
less than Athabasca University (by a factor of about two and a half). 

The British Columbia Open University Independent Study Courses 
As an open learning institution, the BCOU offers independent study 
courses to its student population. The courses rely extensively on 
textbooks, print-based assignments, course units, as well as audio and 
video supplements mailed to students in a course package. 

The main difference between the off-line and online course versions 
was in the the type of student-instructor and student-student 
communication available. For off-line course versions, students were able 
to contact tutors or staff via mail, telephone, or e-mail. Internet access was 
not required, and students were not provided with an opportunity to 
contact other students. Telephone calls were accepted by tutors during 
scheduled twice-weekly office hours and voice mail was provided in 
between office hours. Tutors were required to respond to student queries 
within two working days, as well as to return submitted assignments 
within three working days. Though some students participated in BCOU 
off-line independent study courses without using e-mail, e-mail 
communication has become commonplace and tutor telephone contacts 
have decreased by a factor of at least ten since students started to use the 
Internet to contact tutors. For online course versions, Internet access was 
required. The First Class or WebCT learning management system 
provided access to library and administrative support, to the tutor, to 
student discussions and conferences moderated by the tutor, and to 
'students only' conferences. Online courses still relied extensively on 
textbooks and instructional materials mailed to students in a course 
package but less extensive materials (such as assignments and general 
instnrtions) were provided online. Turnaround time standards for 
conference messages and assignments were the same as for off-line 
courses. Thus the primary difference between off-line and online course 
versions was that the latter (a) Supports group communications 
facilitated by a tutor and (b) Provides students with an online learning 
environment that integrates academic and administrative communication 
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and where other students, staff, and fufors are a couple of 'clicks' away. A 
few BCOU online independenf sfudy courses were also provided in a 
paced formaf. BCOU's course pacing is nof fufor led however, buf rafher 
fhe resulf of superimposing deadlines on an independenf sfudy course 
sfrucfure: 17 courses ouf of 72 courses exisfed in paced formaf as well as 
independenf sfudy formaf af some poinf during fhe 3 years covered by 
fhis sfudy. 


Methods 


Data 

The fargef population was sfudenfs in academic areas (Arfs, Science, and 
Business) who complefed an independenf sfudy disfance educafion 
course in fhe fiscal years 2002/03, 2003/04, and 2004/05 (fhis excludes 
laboratory courses.) During fhaf period, fhe BCOU offered 70 online 
courses and 137 off-line courses, a balance fhaf should enable us to 
compare online courses to off-line courses. The analysis comprises 203 
dafa poinfs based on 70 courses and 6,634 course enrolmenfs. A dafum 
represenfs a unique course delivery mefhod and individual fufor 
combinafion including six or more sfudenfs in a given year. 

Experimental Design 

Course completion rafes were calculated on an annual basis for each 
unique course delivery mefhod and individual fufor combinations. 
Stepwise linear regression was used wifh 16 insfifufional facfors or sub- 
facfors (independenf variables) to identify which were mosf sfrongly 
associated wifh course completion (dependenf variable). Course 
complefion rafe benchmarks were fhen esfablished based on regression 
facfors fhaf were found to be sfafisfically significanf, rafher fhan choosing 
fhem subjectively We used a 0.05 level of sfafisfical significance. This 
mefhodological phase of fhe research projecf is described in aufhor (2007). 

There was an opporfunify fo compare online benchmarks eifher fo 
fheir off-line equivalenf (a paired design fargefing fhe same 70 courses) or 
fo off-line courses overall (137 courses of which only 70 have a direcfly 
equivalenf online version). We opfed for fhe laffer approach because if 
provided greater sfafisfical power for fwo reasons: 1) if nearly doubled 
fhe number of courses (from 70 fo 137) used fo benchmark based on 
insfifufional facfors (aufhor, 2007), and 2) If included all 70 online courses 
as a basis for comparison. Using all 70 online courses would nof be 
possible in a paired design because fhe basic dafum consisfs of a course 
supporfed by a particular fufor in a parficular course version in a given 
year. Consequenfly, fhe sub-sef of courses where off-line and online 
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course versions are supported by the same tutor is considerably less than 
70. Thus, we felt that maximizing sample size for benchmarking purposes 
was fhe besf approach fo compare online courses fo off-line courses. 

Defining Completion Rates 

One of fhe challenges in assessing fhe impacf of virfualizafion on course 
complefion rafes is fhaf resulfs are difficulf fo compare because online 
effecfs are confounded wifh ofher infra- and infer-insfifufional 
differences, including how fo measure complefion rafes (see Howell ef al., 
2004). Though TradifionaT insfifufions and 'open and disfance educafion' 
insfifufions use distinctive grading policies particularly wifh regard fo fhe 
definition of an incomplefe or a wifhdrawal grade, fhe fwo sysfems can 
be linked fhrough fhe concepf of sfable enrolmenf, viz., sfudenfs who 
demonsfrably sfarfed a course. Whereas mosf BC insfifufions identify 
'sfarfers' using a fime-dependenf approach (a cuf-off dafe chosen so as fo 
ensure fhaf sfafisfics are based on sfudenfs who clearly made an affempf 
fo sfarf a course). Open Universifies such as fhe BCOU idenfify sfarfers 
wifh respecf fo oufcome, viz., fhe facf fhaf a sfudenf has complefed some 
graded academic work, regardless of dafe. Through fhe concepf of sfable 
enrolmenf, fhe successful complefion rafe of "sfarfers" links fhe grading 
sysfems of 'open and disfance education' insfifufions and of 'fradifionaT 
insfifufions and fheir comparafive performance (Woodley and Parleff 
1983, Paul 1990, Howell ef. al, 2004). 

BCOU's successful complefion rafes (SCR) were calculafed as follows: 

The total number of students who obtained a passing grade on an ABCD 
scale, divided by the total number of students who started (i.e., the number 
of students who obtained a passing grade on the ABCD scale plus those 
who received an Incomplete or a Fail grade) times 100. 

This course completion rate definition excludes "W" grades as defined 
by 'open and disfance educafion' insfifufions like fhe BCOU, i.e., a grade 
given fo sfudenfs who do nof submif any academic work for grading. The 
BCOU "W" grade is nof included in GPA calculations and as such bears 
no academic penally. 

Coding Institutional Factors 

The insfifufional factors consisf of course factors and fufor factors. The 
course factors are: Course Level, Subjecf Matter, Number of Credifs, Year 
of Delivery, Delivery Pacing, as well as a unique number for each course 
(Course Number). Course Level is coded as follows: 
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Course Level 

Regression Coding 


Subfactor 1 : 

Non-credit (ABE*) = 0 and Credit = 1 


Subfactor 2: 

ABE = 0, and 1 to 4 for university from 100-level to 400-level 
(year 1 to year 4) 

Subfactor 3: 

Introductory course =1 Other courses = 0 (Introductory is first of 
a pair of first- or second-year courses, such as Introductory 
Physics 1 and Introductory Physics 2) 

*ABE = Adult Basic Education 



Subject Matter is coded as follows: Applied Academic areas (Business 
Administration and Business computer courses) are coded as 1. Liberal 
Arts subjects are coded from 2 to 4 according to subjective levels of 
quantitative or scientific contents. 

Subject Matter 

Disciplines 

Regression 

Coding 

Business 

Administration, Business, Workplace Training 

1 

Humanities 

English, French, History, Humanities, Spanish 

2 

Social Sciences 

Economics, Geography, Geology, Political, 
Science, Psychology, Sociology 

3 

Science and Mathematics 

Biology, Environmental Studies, Environmental 
Science, Chemistry, Mathematics, Physics 

4 


Delivery Pacing is coded as 1 (Independent study without pacing) and 
2 (Independent study with pacing, i.e., with assignment due dates). 

The tutor factors are: Course Load (the number of courses supporfed 
by a fufor for fhaf year), Sfudenf Load (fhe number of sfudenfs supporfed 
by a fufor for fhaf year). Age (in 2002), Years of Service af OLA, Gender, 
Place of Residence, Academic Qualifications, as well as a unique number 
for each tutor (Tutor Number). Place of Residence and Academic 
Qualificafions are coded as follows: 
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Tutor Factors Regression Coding 

Place of Residence in 2002/03 1 = Metropolitan: Greater Vancouver and Greater Victoria 

2 = Other urban centres: Kelowna, Kamloops, 

Nanaimo, Prince George 

3 = Rural: All other locations 

Academic Qualifications 1 = Phd, 2 = Master's, 3 = Bachelor's or below 


For more information see Giguere (2007). 

Results and Interpretation 

Identifying Benchmarking Factors from Among Institutional Factors 
We regressed 16 institutional factors on 3 years of online independent 
study course completion rate data. Six regression steps identified 6 factors 
accounting for 42.3% of SCR variation (Table 1). This is a relatively high 
percentage considering that completion rates ranged widely (minimum 
20.0% and maximum 100.0%). In the corresponding off-line course 
benchmark study (author, 2007), eight regression steps accounted for 
46.8% of SCR variation. 


Table 1 : Statistical Regression of institutional Factors on Online Course SCRs (Stepwise 
Linear Regression Method) 


Regression 

Step 

Factors 

F to Enter 

R2 

proportion of variation 
accounted for by 
regression 

1 

Course Level: Subfactor 2 
ABE to 400-level 

69.6 

0.259 

2 

Course Level: Subfactor 1 
Credit vs Non-credit 

41.5 

0.295 

3 

Course Level: Subfactor 3 
Introductory or Not 

37.6 

0.364 

4 

Tutor Gender 

31.4 

0.391 

5 

Course Number 

27.0 

0.409 

6 

Year of Delivery 

23.7 

0.423 
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Overall this regression is comparable to the one for BCOU off-line 
courses (Giguere, 2007) excepf fhaf Subjecf Matter and Tutor Course Load 
did nof enfer info considerafion for fhe online course regression. One 
factor fhaf did nof confribufe to fhe online course regression model 
presenfed here in spife of high expecfafion was Delivery Pacing 
(coefficienf = 0.001, f = .009, p > 0.50). 

The firsf fwo Course Level factors entering af sfep 1 and 2 of fhe 
regression was used fo benchmark online course completion rates for fwo 
reasons. Firsf, fhey accounf for 29.5% of fhe sfafisfical variation before 
fufor specific or course specific facfors enfer fhe regression. Second, fhey 
mafch fwo Course Level facfors used fo benchmark off-line course SCRs 
(Giguere, 2007). 

Sfep 2 of fhe linear regression can fhen be used fo determine online 
course SCRBs according fo fhe general regression formula: 

Y = a -F (b • XI) -F (c • X2) Equation 1 

where XI represenfs fhe slope for Course Level: Subfacfor 2 codes (ABE 
fo 400-level) and X2 fhe slope for Course Level: Subfacfor 1 codes (Credif 
vs Non-credif). 

Equafion 2 generafes fhe following online course SCRBs, shown in 
Table 2 wifh acfuals: 

SCRB = 53.1+ (7.0 • XI) + (14.6 • X2) Equation 2 


Table 2: BCOU Online Successful Course Completion Rates Benchmarks*, by Course 
Level: SCRBs and SCR Actuals (sample size in parenthesis**) 



Non-Credit 

Courses 


Credit Courses 



ABE 

100-Level 

200-Level 

300-Level 

400-Level 

SCRBs 

(from Equation 1) 
SCR Actuals 

53.1 

53.1 (13) 

74.7 

75.8 (107) 

81.7 

76.5 (40) 

88.7 

93.3 (27) 

95.7 

94.9 (16) 


* SCRBs, expressed as a %, for academic courses offerings during 2002/03 through 2004/05 period 
** A datum is based on an SCR percentage for a unique course version and tutor combination for a 
given year 

The fit between the SCRBs (from the linear regression model) and SCR 
actuals is within a few percentage points in each case. Two differences are 
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near 5 percentage points (200- and 300-level courses), one on the high side 
and one on the low side. Actuals suggest three groupings: (a) ABE, 
(b) 100- and 200-level, and (c) 300- and 400-level courses. The benchmarks 
on the other hand show a gradual progression across all levels, since they 
are based on a linear regression model. Nevertheless, the regression 
model organizes online SCRBs congruently and effectively across five 
Course Levels, as was fhe case for off-line SCRBs (Giguere, 2007). 

Other Significant Regression Factors 

Facfors enfering fhe regression in sfeps 3 fhrough 6 accounf for variation 
over and above fhe variation accounfed for by fhe proposed 
benchmarking facfors. Course Level and Subjecf Matter classifications. 
They are Infroducfory Course (coefficienf = -9.8, f = -4.23, p < 0.001; i.e., 
online SCRs are on average 9.8% lower for infroducfory courses). Tutor 
Gender (coefficienf = -6.3 , f = -3.17, p = 0.002; i.e., online SCRs are on 
average 6.3% lower for male fufors fhan for female fufors). Course 
Number (coefficienf = 0.052, f = 2.66, p = 0.008), and Year of Delivery 
(coefficienf = -2.6, f = -2.18, p < 0.031; i.e., online SCRs decrease on average 
2.6% per year). The Infroducfory Course and Gender effecfs are nearly 
double fhose observed in fhe off-line sfudy (decreases of 5.5% and 3.2% 
for Infroducfory Course and Gender respectively, Giguere, 2007). More 
surprisingly however. Year of Delivery has a negafive effecf for online 
course SCRs, whereas if has a posifive effecf for off-line course SCRs (1.9, 
f = 2.8, p < 0.01) (Giguere, 2007). This suggesfs fhaf off-line SCRs increased 
while online SCRs decreased during fhe 2002-2005 sfudy period. The 
averages are shown in Table 3. 


Table 3: BCOU Online and Off-line SCRs (average) Over Three- Year Experimental 
Period (sample size In parenthesis*) 


Year 

2002/03 

2003/04 

2004/05 

Online SCRs 

81.4 (53) 

78.5 (68) 

76.2 (82) 

Offline SCRs* ** 

66.5 (200) 

70.2 (186) 

69.8 (162) 

SCR Difference 

14.9 

8.3 

6.4 

(Online minus Offline) 





* A datum is based on an SCR percentage for a unique course version and tutor combination for a 
given year 

** From Giguere (2007) 
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Thus two components contributed to the BCOU's virtualization 
strategy from 2002/03 through 2004/05, a small SCR gain due an off-line 
SCR increase over fime (aufhor, 2007) and a large SCR gain due fo high 
online course SCRs relafive fo off-line course SCRs. Bofh componenfs 
confribufe foward BCOU's 1997 goal of increasing course complefion 
rafes fo BC sysfem levels. 

To invesfigafe declining online course SCRs from years 2002/03 
fhrough 2004/05, we fesfed for inferacfion between Year of Delivery and 
all ofher significanf regression facfors. Only one inferacfion was 
sfafisfically significanf, i.e., a Year of Delivery by Gender inferacfion 
(f = -3.97, p < 0.0001). Table 4 displays average SCRs by Gender and Year 
of Delivery. 


Table 4: BCOU Online SCRs (average) by Gender, Over Three-Year Experimental 
Period (sample size in parenthesis*) 


Year 

2002/03 

2003/04 

2004/05 

Female 

83.5 (23) 

79.6 (22) 

80.8 (28) 

Male 

79.9 (30) 

78.1 (46) 

73.8 (54) 

Difference 

3.6 

1.5 

7.0 

(Female minus Male) 





* A datum is based on an SCR percentage for a unique course version and tutor combination for a 
given year 

While female fufors mainfained high online course SCRs over fhe 
2002-2005 experimenfal period, male fufor's SCR declined by 6 
percenfage poinfs. The gender gap grew from 3.6 percenfage poinfs in 
2002-2003 fo 7.0 percenfage poinfs in 2004-2005. 

Since fhe fufor gender disfribufion is very similar among online and 
off-line courses over fhe experimenfal period (Table 5; fhis would be 
expecfed in an insfifufion where assefs are frozen and fhe fufor 
population sfable), fhe observed gender SCR difference cannof be 
associafed wifh a gender disfribufion change among online and off-line 
course versions during fhaf period. 
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Table 5: Gender Distribution Among Online and Off-line Course Versions Between 
2002-03 and 2004-05, Wilcoxon Signed Ranks Statistic (Z) 


Year 

Male 

Female 

Online 

Z = -1.177 p = 0.239 

Z = -0.051 p = 0.959 

Offline 

Z = -0.155 p = 0.877 

Z = -0.311 p = 0.756 


We conclude that male tutors are associated with declining online 
SCRs over the experimental period, though the reasons for this are not 
clear. Anecdotal evidence gathered in the context of a performance 
evaluation of Arfs and Science fufors suggesfs fhaf male fufors favor fhe 
insfrucfional approach used in off-line course delivery, an approach fhaf 
consisfs in marking and providing feedback in response fo sfudenf 
requesfs, rafher fhan proactive facilifafion. A fransfer of fhese off-line 
fuforing skills fo online environmenfs by male fufors may have resulfed 
in lower parficipafion levels in online conferences and fhus may have 
resulfed in reducing fhe effecfiveness of BCOU's online delivery by male 
fufors. 

Comparing Online and Off-line SCRBs 

Table 6 expresses resulfs as online SCR gains, i.e., as fhe difference 
befween online and off-line SCRBs. This is obfained by subfracfing SCRBs 
in Table 2 above, from fhe aufhor's (2007) Table 3. 

Table 6: SORB Gains Online (Percentage Point Difference Between Online and Off-line 
SCRBs), By Course Level and Subject Matter 


Course Level 


Subject 

Matter 

Non-Credit 

Courses 

ABE 


Credit Courses 


100-Level 

200-Level 

300-Level 

400-Level 

Business 


0.0 

2.1 

4.3 

6.5 

Humanities 

-1.4 

3.7 

5.9 

8.1 

10.3 

Social Sciences 

2.4 

7.4 

9.6 

11.8 

14.0 

Science and Math 

6.1 

11.1 

13.3 

15.5 
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Table 6 yields three key findings. 

1. SCRB gains online show a strong and coherent pattern across 
Subject Matter and Course Level classifications; 

2. SCRB gains online are positive for all cells but two, and are higher 
the higher the Course Level and the more academic and quantitative 
the Subject Matter; and 

3. SCRB gains online are as large as 15.5 percentage points. This 
represents an increase of up to 21.2% over corresponding off-line 
SCRBs. 

However, since BCOU students can select online or off-line course 
versions at the time of course registration, it is possible that Table 6's 
differences result in whole or in part from the fact that students with 
stranger academic abilities preferentially select online delivery. For 
example students with more prior education could be more successful 
than students with less prior education and choose online course delivery 
more frequently than students with less prior education; if this were the 
case, student behaviour would produce an 'apparent' online SCR increase 
(by apparent we mean SCR increases due to differential student 
distribution among course versions rather than to the benefits of online 
course delivery features), if 'apparent' online SCR gains were to occur, we 
should observe (a) demographic differences among BCOU online and off- 
line student population, (b) a correspondence between online SCR gains 
and significantly different demographics, as per Table 6's results, and (c) 
an over-representation of demographics associated with student success 
in the online student population. Results from a contemporary course 
survey allow us to test this hypothesis. The OLA Course Survey includes 
information about six student demographic characteristics as well as 
course completion rate data. The survey is based on 8,633 respondents 
who finished an online or off-line academic course between September 1, 
2004 and December 31, 2006. 

1) Do demographic differences exist among BCOU online and off-line student 
population ? 

Using a t-statistic for parametric data, Mann-Whitney's z-statistic for 
ordinal data, and statistic for nominal data, we find (Table 7) that two 
out of six student demographics are statistically different among the 
online and off-line student populations (Prior Education, Employment 
Status). Two others yielded probability values between 0.05 and < 0.10 
(Gender and "in BCOU Program") and should be considered for further 
investigation. 
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Table 7: Comparative Demographics for BCOU Online and Off-line Student Populations 
(September 2004-December 2006 Course Survey) 


Student Demographic 

Characteristics 

Offline 

Oniine 

Difference 

Significance 

Age (years) 

32.1 

32.8 

0.7 

T = -1.19 
P = 0.234 

Gender (% male) 

28.8 

32.4 

3.6 

= 6.995 
p = 0.072 

In BCOU Program (%) 

28.9 

46.6 

17.7 

= 6.504 
p = 0.090 

Prior Education“ 

10.1 

10.3 

0.2 

Z = -2.149 
P = 0.032 

Employment Status““ 

1.83 

1.93 

0.1 

Z = -3.19 
P = 0.001 

Region (% metropolitan) 

53.2 

55.0 

1.8 

x2 = 5.046 
p = 0.168 


“Coding based on number of years of education (elementary = 7, grade 12 = 12, 2-year 

Diploma = 14, 4-year degree = 16, etc.) Undeclared = 0 

““Coding: Undeclared = 0, unemployed = 1, student = 2, employed = 3 

Continuing with the two statistically significant student demographics, 
we consider the second expected observation: 

2) Is there correspondence between SCR gains online and statistically significant 
student demographic differences, according to a pattern that is consistent with 
Table 6's results? 

Table 8 displays online SCR gains according to significant demographic 
differences, by Course Level (fop) and Subjecf Maffer (boffom). Only 
"Prior educafion" yields a paffern fhaf is consisfenf wifh Table 6 and a 
posifive answer fo our question. 
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Table 8: SCR Gains Online Across Course Level and Subject Matter Classifications for 
Statistically Significant Demographic Characteristics (Surveyed Population, September 
2004-September 2006) 





Course Level 



ABE“ 

100-Level 

200-Level 

300-Level 

400-Level 

Prior Education^'^ 

- 

1.9 

-8.0 


6.1 

8.8 

Employment Status^'^^'^ 

- 

-4.2 

-7.8 


3.2 

-6.0 




Subject Matter 




Business 

Humanities Social Sciences Science and Math** 

Prior Education^'^ 

4.2 

2.0 


-3.5 


- 

Employment Status^'^^'^ 

0.8 

3.1 


-4.3 


- 


“Insufficient data for online courses 

■•■University course work versus no university course work 

■'■■'■Employed versus unemployed or student 


Continuing with "Prior Education", we consider the third expected 
observation: 

3) Are there proportionally more university-educated students in online course 
versions, and particularly so as Course Level increases? 

We used the Kolmogorov-Smirnov one-tailed test for large samples 
(Siegel, 1956, p. 131) to test whether students with prior university 
education are over-represented in online course versions. We observed 
neither a statistically significant difference (Chi-square value of 3.14, 0.30 
> p > 0.20), nor a proporfional increase along a Course Level classificafion 
(Table 9). 
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Table 9: Proportional Distribution of Students with Prior University Education* Among 
Online and Off-line Student Populations, Across Course Levels (Surveyed Population, 
September 2004-September 2006) 


University level 
Course Work 



Course Level 


100-Level 

200-Level 

300-Level 

400-Level 

Sample Size 

A-Online (% OF total) 

0.342 

0.349 

0.260 

0.333 

1,082 

B- Offline (% of total) 

0.381 

0.309 

0.300 

0.301 

6,840 

Difference (A-B) 

-0.039 

0.040 

-0.040 

0.032 



'Demographic characteristic statistically different among online and off-line student populations and 
exhibiting online SCR gains consistent with Table 6. 


In short, students with prior university education did not choose 
online course delivery in a greater proportion than students without prior 
university education. In the end, we were not able to find evidence that 
BCOU student's participation in online course versions is linked to 
demographic characteristics or academic abilities associated with SCRs. 
This increases the likelihood that the gains online exhibited in Table 6 
reflect the greater effectiveness of fhe BCOU's independenf sfudy online 
delivery model. We suggesf, fherefore, fhaf in mosf BCOU independenf 
sfudy courses, fhe feafures of online course delivery helped sfudenfs 
achieve high SCRs. We also suggesf fhaf fhe effecfiveness of independenf 
sfudy online delivery is greafesf wifh sfudenf populafions engaged in 
upper levels of insfrucfion and in more quanfifafive subjecf maffers. 

Discussion 

Performance of BCOU Independent Study Online Courses Relative to Their Off- 
line Counterparts and the BC System 

To reach a 1997 goal of increasing complefion rafes fo British Columbia 
sysfem levels, fhe BCOU adopfed a virfualizafion sfrafegy based 
primarily on fwinning existing off-line independenf courses for online 
delivery. The BCOU anficipafed fhaf re-purposing exisfing off-line 
courses online would allow if (a) fo esfablish a sfrong online presence, 
rapidly and cosf-effectively, and (b) fo increase sfudenf success while 
profecfing fradifional off-line enrolmenf. Since BCOU assefs (sfaff. 
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curriculum, resources) were frozen shortly thereafter (from 2002 to April 
1, 2005) in preparation for its acquisition by another educational 
institution, and since the BCOU offered a large number of equivalent 
courses in online and off-line versions at that time, the BCOU represents 
a 'natural' experiment to assess the impact of online delivery on 
independent study course completion rates. To that effect, we 
benchmarked BCOU's online and off-line course completion rates for the 
academic years 2002/2003 through 2004/2005 using a standard method 
(Giguere, 2007). 

The benchmarking method consists in regressing up to 16 institutional 
factors on successful completion rate (SCR) data organized according to 
unique course delivery method and tutor combinations, for each of the 
academic years 2002/2003 through 2004/2005. The regression method 
accounted for over 42% of the total variation. Because Course Level and 
Subject Matter classification loaded in the early steps of regression before 
tutor specific or course specific factors came into play, it was possible to 
use Course Level and Subject Matter factors to benchmark online and off- 
line course completion rates congruently and effectively. The results 
indicate that the benefits of BCOU's virtualization has two components, 
an off-line SCRB increase over time that is possibly due to increased e- 
mail usage in off-line courses over time (author, 2007), and high online 
SCRBs relative to off-line SCRBs (this article). Online SCR gains were 
strong and followed a consistent pattern: they were particularly large for 
upper-level courses and for courses in academic and quantitative 
disciplines (SCRB gains of up to 21.2%; Table 6). 

By comparing our results and the author's (2007) to BC system 
performance statistics provided by course level early during the BCOU 
embarked on its course twinning virtualization strategy (British 
Columbia 2000a, 2000b), we can check on BCOU's 1997 goal to increase 
course completion rates to BC system levels (Table 10.) 
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Table 10. SCRs for the BC College, Institute and Agency System and BCOU Off-line 
and Online Academic Courses for 2002-2005 period (sample size in parenthesis*) 



ABE 

Lower Level 

Upper Level 

BC College, Institute and 




Agency system, 1998 

72 

79 

93 

BCOU online SCRs, 2002/05* 

53.1 

76.0 

93.9 


(13) 

(147) 

(43) 

BCOU offline SCRs, 2002/05* 

49.2 

70.0 

83.3 


(103) 

(336) 

(109) 


*A datum is based on an SCR percentage for a unique course version and tutor combination for a 
given year 


Though BCOU off-line course SCRs fall well shorf of BC sysfem 
performance af all levels, BCOU's online courses SCRs fare well in fhe 
credif course area. BCOU's course fwinning sfrafegy had a negligible 
effecf on SCRs for non-credif courses, i.e., in fhe ABE area. BCOU online 
SCRs exceed BC sysfem sfafisfics for upper-level credif courses and fall 
slighfly shorf for lower-level credif courses. These resulfs demonsfrafe 
near equivalency befween BCOU's independenf sfudy online credif 
courses and face-fo-face courses offered by BC public insfifufions, when 
using SCR sfafisfics based on a concepf of sfable enrolmenf. Our resulfs 
are in agreemenf wifh ofher sfudies fhaf found high complefion rafes for 
online course delivery, including claims of 'equivalency' wifh face-fo-face 
course delivery (Card & Horfon 1998, Fredda 2000a, 2000b, 2000c, 
Johnson ef al. 2000, Machfmes & Asher 2000, McDonald & Gibson 1998, 
Rosbeig 1997, Russell 1999, Sener 1996). Our sfudy specifically 
demonsfrafes fhaf online independenf sfudy course designs can also 
achieve near equivalency wifh comparable face-fo-face courses. 

Assessing BCOU’s Virtualization Strategy 

Our resulfs demonsfrafe fhaf fhe BCOU virfualizafion sfrafegy achieved 
parf of fhe goal of increasing course complefion rafes "by an amounf 
sufficienf fo ensure fhaf wifhin fwo years (i.e., by March 1999) fhey meef 
or exceed fhe minimum as esfablished by fhe ofher insfifufions in fhe 
Brifish Columbia College, Insfifufe and Agency sysfem (confrolling for 
program area and using sysfem-wide key performance indicafors.)" OLA 
(1997). Though fhis goal was nof achieved on schedule (i.e., by March 
1999), our sfudy suggesfs fhaf online credif courses achieved fhis 
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performance level as they were launched. Since online SCR gains can be 
large (particularly at the upper-level and in academic and quantitative 
subject matters), since our experimental design is robust (course twins are 
nearly equivalent and the institution's assets were frozen at the time of 
this study), and since BCOU student participation in online course 
versions is not linked to demographic characteristics associated with 
SCRs, we suggest that BCOU online course delivery features themselves 
helped students achieve high SCRs. 

if online course delivery features per se help independent study 
students succeed (rather than being the result of students self-selecting 
themselves into online delivery according to ability,) it is difficult for the 
institution to justify maintaining twin-course versions to protect access to 
courses for a traditional audience. If on the one hand SCR gains online 
were the result of students re-distributing themselves among course 
versions according to academic ability, overall institutional SCR statistics 
would represent a zero-sum game and offering twin online and off-line 
course versions would make sense because this provides traditional 
audiences with more choice and more time to become familiar with, and 
adopt online course delivery. If on the other hand the online delivery 
features themselves help students achieve high SCRs, there exists an 
ethical imperative (1) To inform students of the online advantage so 
students can make well informed course selection decisions, and (2) For 
the institution to strive to move away from 'course twinning' toward 
online delivery. 

Since the BCOU has invested considerable assets in developing a twin 
online and off-line platform however, a mixed virtualization strategy 
would make sense on an interim basis. "Online delivery offerings only" 
should be considered in consultation with faculty and students, 
particularly where SCRs are close to BC system statistics and online 
benefits are substantial (3rd and 4th year courses in quantitative 
disciplines for example.) On the other hand twin off-line and online 
delivery could be maintained where SCRs are similar and close to BC 
system performance statistics, if the cost-benefits of supporting two 
course versions warrant, or in areas of special needs. In the ABE 
programming area where both online and off-line SCRs are considerably 
lower than BC system performance statistics, new delivery approaches 
should be contemplated, such as tutor-led paced delivery. 

Other institutions that make extensive use of independent study 
delivery should review their completion rate statistics in the light of this 
study and research the suitability of Subject Matter and Course Level 
classifications to optimize their course delivery platforms. 

In closing, let us review BCOU independent study course delivery 
features to help identify which ones might have contributed to high 
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online SCRs. Although this study was not designed to identify benefits 
among alternate online delivery features, BCOU's off-line and online 
delivery models are fhe producf of a rigorous insfrucfional developmenf 
process and off-line and online course versions exhibif consisfenf 
disfinguishing feafures. The main difference befween off-line and online 
course versions is in sfudenf-insfrucfor and sfudenf-sfudenf 
communicafion fhaf is available, and how fhaf is mediafed and facilifafed. 
Off-line courses do nof provide group communicafions and are based 
primarily on e-mail fufor supporf (fhe use of felephone supporf has 
become relatively infrequenf). If is a reacfive communicafions model fhaf 
requires fhe sfudenf fo inifiafe communicafion around specific problems 
and issues. In fhe online courses, sfudenfs are provided wifh an online 
learning environmenf fhaf infegrafes fhe various fypes of insfrucfional 
and adminisfrafive communicafion. Through fhe learning managemenf 
sysfem (Firsf Class or WebCT) sfudenfs have direcf access fo library and 
adminisfrafive supporf, fo fhe fufor, and fo sfudenf discussions and 
conferences moderafed by a fufor (sfudenfs, sfaff, and fufors are a couple 
of 'clicks' away.) Tutors are more proactively involved in assisfing 
sfudenfs. This suggesfs fhaf fhe infegrafed learning environmenf of fhe 
online course model yields subsfanfial benefifs fo credif courses 
independenf sfudy sfudenfs, parficularly af higher levels and in more 
quanfifafive subjecf maffers. Supplementing prinf -based independenf 
sfudy course maferials wifh e-mail fufor supporf (an approach fhaf we 
refer fo as off-line in fhis sfudy buf fhaf is often considered as online 
delivery by some insfifufions) may improve sfudenf success over 
fradifional prinf-based delivery (wifh fufor felephone supporf) however 
fhis sfudy suggesfs fhaf SCRs will be even higher if sfudenfs are provided 
wifh an infegrafed online learning environmenf. 
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